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1. Introduction 



In a classical stratified sampling, a population is split into a fixed number L of 
strata and in each of them a subsample is observed. Various bootstrap estimators 
of the mean have been studied as the strata sizes are finite [HI [18] or infinite 
[2113]. In a two-stage cluster sampling the population consists of clusters of units, 
first some clusters are sampled and then units are sampled within the selected 
clusters. Several two-stage boostrap methods have also been studied in the case 
of a finite number of finite clusters drawn without replacement [161 118] . Here, we 
consider a two-stage cluster sampling with an infinite number of clusters. The 
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general setting is the following: A general population is subdivided into a large 
number of independent populations (or clusters) which cannot be all observed. 
We consider the problem of estimating some parameters of the distribution 
function of a variable X on individuals in the general population and we assume 
that the realizations of this variable in the different populations are independent 
and identically distributed. The observations of X on individuals within the 
same population are supposed to be independent and identically distributed 
conditionally on the selected population. 

Here the parameters of interest are the means of X, the variance of the 
means of X in the different sub-populations and the variance of X and we focus 
on the behavior of bootstrap estimators. The normalized and the studentized 
estimators of the mean are asymptotic Gaussian as the number of sampled 
populations tends to infinity and the variances are consistently estimated. Three 
bootstrap procedures are studied, 

Bl Sampling the populations with replacement in the set of the observed pop- 
ulations, then taking the original observed data of these sampled popula- 
tions, 

B2 Taking each of the observed populations and sampling the individuals with 
replacement in the set of the observed data within each population, 

B3 A two-stage bootstrap cluster sampling that is a combination of the first 
two procedures. Here the populations are sampled with replacement from 
the set of the observed populations, then individuals are sampled with 
replacement within each population. 

An application and simulation of the Bl sampling for a large number of 
populations was presented in [ITJ [12] for specific parameters in forestry. Boot- 
strap sampling with a large number of dependent variables looks like a stratified 
sampling, where individuals and variables may be drawn in the bootstrap pro- 
cedure. Only the bootstrap resampling of individuals is relevant due to the loss 
of dependence between dependent variables of an individual when they are not 
jointly resampled, as in [13l HH [15] with an application to medical data. 

The asymptotic properties of the mean bootstrap estimator in the classical 
i.i.d. bootstrap were studied by many authors (e.g. p] O El Hi]). They have 
been extended to some cases of independent but non identically distributed vari- 
ables and to functional results in El 01 OH HE] • We prove properties for some of 
the three bootstrap procedures when the number of strata and the sub-sampling 
sizes tend to infinity, under a condition for their respective rate. The results are 
quite different from those of [H El [TBI [18] for means of fixed sub-populations and 
with resampling of all the populations (B2 sampling scheme), equivalent results 
are established in proposition 2.2 for the mean of the sub-population means. 
Though the bootstrap resampling scheme Bl is consistent for the mean param- 
eters n and fik of the general population and sub-populations, it is not for the 
variances of their estimators fi, and and other bootstrap estimators of the vari- 
ances are necessary to obtain asymptotically normal estimators. With random 
populations, the bootstrap sampling scheme B2 gives consistent and asymptot- 
ically normal estimators for all the parameters with weighted bootstrap. Under 
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a weighted bootstrap resampling scheme Bl, the estimator of the global mean 
and its variance achieve the usual properties. Under sampling scheme B3, the 
bootstrap estimators of /j, and [i k and the variances are not consistent. The rate 
of the bootstrap approximation is given for the weighted resampling scheme 
Bl. As the variance of the estimators splits into within and between-population 
variances of different orders, the asymptotic results are quite different from the 
usual results. 

2. Empirical estimation in a subdivided population 

A variable X is observed according to the following two-stage cluster sampling: 
a sample of K populations is selected among a large number L of populations 
and, in the fc-th population of unknown large size, we consider n k independent 
observations of the variable of interest, X k i,i < n k , k < K. Let N — J2k<K Uk 
be the total number of observations, we suppose that N and rife, for each k, 
increase with K. The estimators of the global population have then equivalently 
indexed by N = Nk or K. 

Conditionally on the fc-th population sampling, the variables X k i,i < n k , 
have the distribution function F k and the distribution of the variables Xki in 
the general population is F including the distribution functions of the L sub- 
populations. Denote by E the expectation with respect to the sampling of the 
populations and by Ek the conditional expectation in the fc-th population, fx k 
the conditional mean of the Xki 's in the fc-th population = EXki = E[ik . 

If i?|A| 2 < oo, the variance of X is denoted V in the general population 
and the random variance of the Xki 's conditionally on the sampling of the fc-th 
population is Vk — Ek(Xki — Hk) 2 ■ We also denote by 7 = E(pik — n) 2 the 
variance of the independent random variables fik, and a 2 = EVk- The variance 
of the XkiS is 



The variables Xki and Xkj, i 7^ j, are independent within the fc-th population 
but they are dependent in the general population, with Cov(Xki, Xkj) = 7. The 
variables Xki and Xy, fc 7^ I, are independent for any i and j. A random effect 
linear model having a nested error structure can be used to describe the data, 



with a k = Hk - A* and u ki = X k i - Hk, where E k u k i =0, Var k u k i = V k and 
E k (ukiUkj) = for i ^ j, Ecik = 0,Varcik = 7 and E(a k ai) = for k I. 
Within the fc-th population, \ik and Vk are unbiasedly estimated by 



V = a 2 + 7. 



(1) 



X k i = A* + a k + u^, i = 1, • • • , n k , k = 1 . . . , K, 




i<nk 
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For the global population, \x is estimated by the empirical mean /^at of the 
X's. Several boostrap estimators of /z will be proved to converge to the empirical 
estimator fl' K of the mean of the intra-populations means. For the general mean, 
the estimators are denoted 



fJ-N 



h E E = E (3) 



N ^— ' ^— ' ^ TV 

k<K i<n k k<K 



Note that with random populations, both jjjv and jS^ have the expectation jjt 
since -E/ifc = /i for every fc. It would not be with fixed populations. We have 
Var k Jl k = Sfct/ifc - fi k ) 2 = % X Vfe and 

Var^ = E^/J/. - ^i) 2 = 7 + -. 

The variance terms a 2 , V and 7 of the variable X are estimated by 



>N 



In 



\ ^ = ^ E E ( x « - ^) 2 > 

fc<if fc<K i<n fc 

(?* - &) 2 - )? E — ~r E - ^-) 2 ' ( 5 ) 



V)v = 7iV + Sjv- (6) 
The variances of the estimators are 

S 2 N :=Varfl N = ±(J2n k EV k ) + 7 n* = ^ + 7 n*, (7) 

k 

S K :=Var» K = -(7 + - ^ — ) = -(7 + (8) 

where £"14 satisfies (JXJ) , n^ 1 = iT -1 J2 k n^ 1 < 1 is the harmonic mean of the 
Ufc's and n* — N~ 2 ~^2 k n| < TV -1 maxj^-L n k with n* = -ftT -1 when constant 
for every fc, then the estimators are identical. The variance of jl' K is smaller 
than the variance of n when 7er~ 2 > {(nK)^ 1 — N" 1 }^* — K^ 1 }^ 1 which may 
happen even with K = 2. If n k = c k K a with < a < 1 and bounded constants 
Cfc for every fc, then Sn < S' K . The variance of both estimators split into a 
within-populations variance and a between-populations variance, Vari nter fiN = 
N~ X (J 2 and Vari n terp-N = 1*7. 

By {U-© an d with n k — 0(K a ), asymptotically unbiased estimators of 
Varji' K and VarfiN are 

Varfl K = \ ^2(fi k -tf K ) 2 , (9) 

k<K 
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Estimators of the within and between-populations variances follow as 
VarinterpN = n*j N , Var intra fi N = — ^ ^ n k V k , 

k<K 

t> In f> ^, 1 Vk 

k<K nk 

Under a convergence rate for the sub-sample sizes, the estimators are all con- 
sistent, and the variances of order K~ x are larger than the variance of fL^. The 
studentized statistics, as well as the normalized statistics converge in distribu- 
tion: 

Proposition 2.1 // E\X\ 2+S < oo for some S > and n k is of order K a , 
< a < 1/2, for k = 1, . . . , K , then /2jv converges a.s. to fi as N — ^ oo, and jl' K 
converges in 'probability to fi as K — > oo, VarjiN — VarjiN and VarJ2' K — Varf2' K 
converge in probability to 0, (Varf2ff)~ 1 ^ 2 ('fiN ~ A 4 ) an d (Varf2' K )^ 1 / 2 (J2' K — //) 
converge in distribution to standard Gaussian variables. 

Proof. J2n ~ t 1 and J2' K — fi' are the means of the (weighted) independent variables 
having zero mean and variances of main order n* and K respectively. With 
n k = 0{K a ) for k = 1, . . . ,K, K = 0(N 1 ^ 1+a ' ) ), n* = 0(N~ 1 K a ) = 0(A" _1 ), 
and Kn = 0{K a+1 ), the variances S% = N^^+jn* oifi N and S'g = K -1 {7+ 
a 2 /n} of f£' K (© and ©) are such that jN- x K a < S% < N^K 01 ^ + o(l)) 
and #- x 7 < Sg < K' 1 ^ + o(l)). Then ^at ^Ajv < 00 whi c h 

ensures the 

a.s. converge of the estimator j5jv by the Borel-Cantelli lemma. 

The boundedness of the variances S N and S3 imply the following Lindeberg 
conditions 



-L££(^- M ) 2 i { | X ^ 



-j*|>eS W } * U 



and the weak convergence of the normalized variables is a consequence of a CLT 
0- ■ 

If we consider ]±k — K^ 1 J"\ —1 /Zfe, similar results hold for ]~l' k —J1k, with vari- 
ance cr^ifn) -1 = VarintrapKi an< ^ f° r M — A 4 with variance 7if~ 1 = Uari„ teT ./I^-. 
The result in the proposition below is similar to the convergence property in a 
stratified population with fixed strata [4]. 

Proposition 2.2 Under the conditions of the proposition \2.1\ , fi' K — \±k and 
p,K — A 4 ' converge in probability to zero, K 1 / 2 ^^ 1 1 Vk)~ 1 ^ 2 (J2 / K — fi) and 

K 1 ^ 2 ^f K 1 ^ 2 (p,K — A 4 ') converge in distribution to standard Gaussian variables. 
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3. Bootstrap estimation 

The observed data set is denoted by X and we consider the three bootstrap pro- 
cedures described in the introduction. For each of them, E* and Var* denote 
the mean and variance for the bootstrap sampling distribution conditionally on 
X, without any distinction of the specific distribution, F k is the empirical sub- 
distribution function of the fe-th population. 



3. 1 . Bootstrap sampling of individuals 

We consider a bootstrap sample which consists of the sub-samples set of size 
n k for the fc-th observed population, {X ki )i< nk , where the variables have 
the distribution F k , i < n^, k < K and the K populations are independent and 
considered as fixed. The bootstrap version of the previous estimators are written 

K = = n l x E X k*> 

i<n k 

v*n = ^EE x « = N ~' E 

k<Ki<n k k<K 

Tk = ^Es, 

k<K 

they satisfy E^* k = ju fc , E*fI* N = fix, = Ji' K , E^* k 2 = (n k -l)n k 2 V k + ji 2 k . 

The variances of fl* N and ji 1 ^ reduce to intra-population variances given by 

Var*/j k = j — Vki 



Var*$* N = Y ( n k - l)Vfc, 



k<K 



k<K 



Finally, let 



v k - ( „ fe "_ fc i)2 E (*« - 

i<n k 



it is an unbiased bootstrap estimator of V k and unbiased bootstrap estimators 
the variances of p,* N and fl'^ follow. 

Then bootstrapping on independent individuals only with fixed populations 
(resampling scheme B2) as in the classical stratified sampling leads to estima- 
tors of [i having variances asymptotically equivalent to the within-populations 
variance of the estimators /2. All variables ,i < n k ,k < n are independent 
conditionally on the sample X and 1Z* N — j3jv is the mean of the N independent 
centered variables (X^ —J2k)- The next convergence results are then proved as 
Proposition ^. li 
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Proposition 3.1 If E\X\ 2+S < oo for some S > and n k is of order K a , < 
a < 1/2, for k = 1, . . . , K , then conditionally on X , (J2* N — /2/v) and (ju^ — 
converge o.s. to zero, (Var* ltra ^)- 1 / 2 (^ - fi N ) and (Var*^.^'*)- 1 / 2 ^'* - 
fi' K ) converge weakly to standard Gaussian variables. 

3.2. Bootstrap sampling of the populations 

A sample of K populations is drawn uniformly from the observed population 
set, each of them having the probability K~ x , and the variables are defined 
as the observed values of the variable X in these sampled populations, according 
to the sampling scheme Bl. We get jEt£ = fli with probability K -1 , for every 
k,l < K, and E^l — j2' K and E^JT 2 = K~ 1 J2i<k$ f° r ever Y k. It follows 
that both fi'x and J1* N have the bootstrap mean J2' K and the bootstrap variances 
are 

i{ _ ^ ^ 

Var*jl* N = N^ 1 2J n k (J2 k - ft' K ) 2 = n* — Varfl' Kl 

k<K 

Var*jl% = i (#* - Mk) 2 = -^-^ ar MK- 
fe<K 

The usual bootstrap estimator of the variance of ju^ is K~ 1 S'£ where = 
K~l J2k<i<(fik ~V-'k) 2 i s the bootstrap variance of the ju^'s, it is a strongly con- 
sistent bootstrap estimator and E*S'^ = (K — l)i^ _1 V arJ2' K . We now get boot- 
strap estimators of [i having a variance asymptotically equivalent to K~ 1 Var'Jl' K 
as K — > oo and the asymptotic behavior of jl^ is similar to that of J1' K : 

Proposition 3.2 Under the conditions of Proposition \3~$\ and conditionally on 
X , K x l S*^ X ($k ~ Mk) converges weakly to a standard Gaussian variables. 

As the uniform population sampling is not relevant for //jy, let us consider a 
sample of K populations drawn with probabilities n^N^ 1 for population k = 
1,...,K, and the variables X^ are defined as the observed values of the variable 
X in these sampled populations. The means of the bootstrap estimators are 
= E^fi* N = /2jv, and for I = 1, . . . , K their variance are 

1<K 

Var*Jl* N = n k (fl k -fi N ) 2 . 

k<K 

An unbiased bootstrap estimator of the variance of /tjv is also an estimator of 
this variance: 

N = n* - 1 ^ nfe ^ /ifc ~ ^ ' 

k<K 

a weighted bootstrap variance of the jujjl's. 
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Proposition 3.3 Under the conditions of Proposition \3^4\ and conditionally on 
X , S^~ 1 (p^ f — Pn) converges weakly to a standard Gaussian variable. 

3.3. Cluster bootstrap sampling 

For the B3 bootstrap sampling, K independent populations are drawn uni- 
formly from the observed population set and the bootstrap sample consists of 
sub-samples of size ni — 1 and distribution Fi if the the fc-th bootstrap population 
is the l-th observed population. Denote Y t * bootstrap variables having the distri- 
bution Fi and X ki bootstrap variables having the distribution Fi with probabil- 
ity K , for every l,k = 1, . . . , K. The bootstrap sampling distribution is then 
F = K- 1 J2i<k Fu E*X* kl = V! K and = ^ with probability 

K -1 , for every I and k = 1, . . . , K. We obtain E*ji* k — E^fr = fi' K = E*fi* N , 

for every k and 

Var*jl% = - Mr:) 2 + — } = — TPT^^k + Vari„tra,P K 

Ze- 
is unbiasedly estimated by 

The behavior of fi* N also depends on /x^ in this procedure, with 
VarSk = (»* ~ l)K 2 Var^% + (£g - 

and jujy cannot be simply estimated in this way. For pf K , the variance of the 
bootstrap estimator p!^ in the two-stage cluster procedure is equivalent to the 
sum of the total and within-populations variances of fl n as N and K — > oo, it 
is therefore unsuitable, and we get 

Proposition 3.4 Under the conditions of Proposition and conditionally on X, 
(Var*/2jf ) _1 / 2 (/2j£ — p' K ) converges weakly to a standard Gaussian variable. 

3-4- Second order asymptotics 

It appears that among the studied bootstrap procedures, only the second re- 
sampling scheme of B2 provides consistent estimators for p, and the variances 
of the estimators. Further expansions prove that the bootstrap estimator of p 
satisfy the classical properties of bootstrap estimators in the independent case 
[U [8l [lOl [17]. The variances are denoted Sjy = Varpjy, S% = VarpM and 
s *n = Var*p* N . 
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Proposition 3.5 If E\X\ 3 < oo andnk = c^K a , for some positive and bounded 
Ck and < a < 1/2, for k = 1, . . . , K , then the bootstrap estimators based on the 
population sampling procedure, with probability n^N^ 1 for population k, satisfy 
a.s. 

limsupsup J ft: 1/2+2Q |P{S , w 1 (/iAr - n) < x} 

~P*{S N {fi N - n N ) < x}\ < . 

where C is the constant of the Berry-Esseen bound. 

If E\X\ e+s < 00 for some 5 > and the d.f. Fk are continuous for k < K. 
then a.s. 

limsupsupi^+^lP^Oijv -n)<x}- P*{S* N -\-p* N - fi N ) < x}\ = 0. 

K—+00 x 

Proof. The variances satisfy S 2 N = K' 1 ^ + o- 2 0(K~ a )), S% = S% + o p (l) a.s. 
under convergence rate of the n&'s. The bootstrap variance S 2 N is estimated by 
Sff in l3.1[ and Sff = S% + o(l) a.s. The Berry-Esseen theorem for a weighted 
sum of independent variables with varying distributions applies for both ftpf — [x 
and Ji N - fiN- Using the expansion S N 3 = K 3 / 2 {^ 3 / 2 + 0(K~ a )} and the 
inequality E\N- l n k (p h - /i)| 3 < A^~ 3 ^| £\< njb (X ki - n)\ 3 < 4N~ 3 n k E\X - ,j| 3 
for each k, we have ^2 k E\N~ 1 n k (fl k — m)| 3 < <±N~ 2 E\X — fj,\ 3 and, uniformly 



sup \P{Sj? £ ^ k -I*)<x}- Hx)\ < CS N 3 Ei^k A'l) 3 

x k<K k 

< K-^-^j-^iCElX - n\ 3 (1 + o p (l)). 
For the bootstrap estimator, a.s. conditionally on X uniformly in x, 
E*N~ 3 nl\K ~ $n\ 3 = N- 3 J2 n*lfo - m| 3 , 

k k 

sup £ ^(K - ?n) <x}- *(*)! < cs n 3 J2e*\^(K - m)\ 



k<K k 
k 

and the first result follows. 

The second part of the proposition is a consequence of Edgeworth expansions 
of fi N - H and fi N - $ N , with S N 3 = K 3 ' 2 ^- 3 ' 2 + 0{K~ a )}. Uniformly in x, 

P{5^ 1 (/2jv - fx) < x} = 



A- 
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with p(x) = 2x 2 + 1. The expansion for the bootstrap estimator is 
P*^)- 1 ^ - far) < x} = + -4t2^(^ - /2 A ,) 3 / 2 {p( a ;) ( /)(x) 

+o(l)} = $(as) + -^N~ 3 Mfo ~ v)\*{p{x)J>{x) + o(l)}, 

with Sn converging to Sjy and the sum of third moments converging to their 
expectation at the rate AT -1 . ■ 



4. Discussion 

The model studied in this paper is the model of means of a variable Xi having a 
mixture distribution of L d.f. Fx, . . . , Fl, defined as F^(x) — P{Xi < x\i £ Vk}, 
for the k-th sub-population Vk- The variable Xi has the d.f. F = Sfc =1 Pfc-Pfc, 

with p k = P{Xi e V k }. 

For a stratified population with a fixed number of strata, [5] have proved 
convergence of the bootstrap Studentized estimators under the condition of sub- 
sample sizes rik of the same order O(N), they have proved in particular that the 
distribution function of (Var'fi^)^ 1 / 2 £*(jut — Mfc) 3 is an approximation of 
(Varp.k)^ 1 ^ 2 E(jik — Mfc) 3 - Their model is different from the random model 
considered in this paper under the slower convergence rate n k = 0(K a ), < 
a < 1/2 and with weighted estimators. In proposition 13.51 the order of the 
approximations is K 1 ' 2 ^ 2 ", that is stronger than the usual results. 

Inversion of several expansions improve coverage probabilities and have been 
compared to bootstrap confidence intervals deduced from the approximation of 
the distribution of Studentized statistics, these numerical improvements were 
discussed in p] [10] and in many other papers later. Here all the estimators 
are differentiable transformations of moment estimators that admit Edgeworth 
expansions which provides a second order approximation of the distribution 
function of S^j (j£t/v — by its inversion, 

p{s N \t N - M ) < x - (i - ^ E^W*-/^ = Hx) + 0(x -( Q+ i/2)) 

but the order 0(i^ _< - Q+1 / 2 ^), coming from the expression of Sjv, differs from 
0(K~ X ), the expected term after a correction term 0(K~ x l 2 ) in a one-stage 
sampling. For the studentized statistic, the correction is similar, with 1 — x 2 
replaced by 1 + 2a; 2 . 
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